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PREFACE 
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amendments.  The  objective  of  the  task  was  to  help  develop  quantitative  relationships 
between  aviation  units'  ability  to  perform  their  missions  and  the  levels  of  training  resources 
available  to  them. 
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I.  INTRODUCTION 


« 


This  paper  develops  quantitative  relationships  between  how  much  aircrews  have 
flown  and  how  well  they  perform  important  aspects  of  their  missions.  In  part,  the  research 
described  here  is  in  response  to  concerns  voiced  by  the  General  Accounting  Office  and 
skepticism  displayed  by  Congress  about  the  impact  of  cuts  to  the  flying  hour  programs  of 
the  services  [1]. 


An  earlier  paper  described  the  small  body  of  existing  literature  that  has  developed 
such  relationships  [2].1  It  reached  the  following  overall  conclusions: 


Quantitative  relationships  that  support  the  proposition  that  more  flying  results 
in  measurably  better  performance  have  been  developed  for  both  the  Air  Force 
and  the  Navy. 


Additional  flying  appears  to  improve  aircrew  performance  in  two  ways.  In  the 
short  ran  it  hones  skills  and  prevents  their  deterioration.  In  the  long  run  it 
permits  the  attainment  of  a  higher  level  of  mastery  that  is  reflected  in  better 
performance.  None  of  the  existing  analyses  that  were  reviewed  fully  captures 
both  of  these  effects. 


•  Data  exist  to  develop  additional  links  between  flying  hour  activity  and 
measures  of  operational  performance  for  a  wide  range  of  aircraft.  Additional 
research  to  build  such  links  should  be  developed. 


Our  purpose  here  is  to  report  the  results  of  new  statistical  analyses  that  examine 
uoili  the  long-range  and  short-range  effects  of  flying  houis  on  performance,  and  to  briefly 
review  efforts  that  have  been  initiated  by  the  services  to  link  flying- hour  histories  to 
indicators  of  performance. 


Three  empirical  investigations  were  undertaken.  The  first  examined  the  quality  of 
landings  aboard  aircraft  carriers  for  F-14  and  A-7  aircraft.  The  second  focused  on  the 
accuracy  with  which  Marine,  aviators  dropped  bombs  from  AV-8B,  F/A-18,  and  F-4S 


1  Examples  of  quantitative  relationships  that  have  been  developed  statiscally  include  the  findings  that  a 
10%  charge  in  recent  flying  hours  can  be  expected  to  change  the  number  of  unsuccessful  attempts  to 
land  aboard  aircraf  t  carriers  by  10%  [3],  and  that  a  doubling  of  pilot  experience  is  associated  with  13% 
greater  bombing  accuracy  [4J. 
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aircraft.  The  third  draws  on  the  performance  of  F-14  fighters  during  opposed  air-combat- 
maneuvering  exercises  on  an  instrumented  range. 

The  nature  and  results  of  these  investigations  will  be  discussed  in  turn. 
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II.  ANALYSIS  OF  CARRIER  LANDINGS 


A.  DATA 

Information  was  gathered  on  4,35 1  carrier  landings  performed  by  60  pilots  flying 
F-14s  and  A-7s  in  Carrier  Air  Wing  7  between  June  1985  and  October  1987. 2  Data  were 
obtained  from  squadron  records  on  how  many  flying  hours  each  of  the  pilots  had 
accumulated  in  his  career,  and  a  running  total  of  how  much  recent  flying  each  had  done 
was  developed.3  The  measure  of  performance  that  we  analysed  was  based  on  carrier 
landing  grades. 

Every  carrier  landing  attempt  is  graded  by  a  landing  signal  officer  (LSO).  The 
LSOs  are  highly  trained  and  the  grading  procedures  that  they  follow  are  tightly  specified. 
Squadrons  monitor  landing  grades  as  an  indicator  of  pilot  proficiency  and  as  a  guide  to 
where  remedial  attention  should  be  placed.4  Seven  landing  grades  are  possible: 

•  0 — dangerous. 

•  1 .0 — wave  off.  The  pilot  was  instructed  not  to  try  to  land.5 

•  2.0 — "no  grade."  A  landing  was  made,  but  landing  technique  was  deemed 
faulty,  though  not  dangerous. 

•  2.5 — bolter.  The  aircraft  touched  down,  but  it  did  not  catch  an  arresting  wire 
and  was  forced  to  take  off  again.6 

•  3.0 — fair  pass.  Some  errors  were  made,  but  overall  technique  was  not  faulty. 


*  Landings  performed  by  E-2Cs  were  also  analyzed.  No  consistent  relationships  between  landing 
performance  and  either  short-term  or  long-term  flying  hours  were  found  for  this  aircraft.  This  may  be 
related  to  the  presence  of  two  pilots  in  the  E-2C.  There  is  reason  to  believe  that  crews  are  made  up  in  a 
way  that  allows  the  deficiencies  of  one  pilot  to  be  compensated  for  by  the  strengths  of  the  other.  This 
would  tend  to  mask  the  link  between  flying  hours  and  proficiency. 

3  We  had  hoped  to  also  get  information  on  both  recent  and  long-term  experience  in  simulators,  but  it  was 
not  available. 

4  To  the  extent  that  poor  landing  performance  is  used  to  allocate  flying  time  to  those  who  need  it  (he 
most,  one  would  expect  an  inverse  correlation  between  recent  flying  hours  and  landing  grades. 
Observation  of  a  positive  correlation  means  that  the  effect  of  additional  flying  in  building  proficiency 
outweighs  this  selection  effect. 

5  If  a  wave  off  is  self-initiated,  or  if  it  is  given  for  reasons  related  to  air  traffic  control,  the  pass  is  graded 
as  a  2.0. 

6  The  LSO  may  assign  a  higher  grade  if  he  believes  the  bolter  was  not  caused  by  pilot  technique. 
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•  4.0 — OK  pass.  A  successful  landing,  no  noteworthy  mistakes  were  made. 
This  is  as  good  a  grade  as  a  pilot  can  reasonably  expect 

•  5.0 — rails  pass.  A  perfect  landing,  like  you're  coming  in  on  rails.  This  grade 
is  very  rarely  given. 

We  decided  not  to  analyze  the  numerical  values  of  the  landing  grades.  There  is  no 
reason  to  believe  that  a  "no  grade"  is  twice  as  good  a  performance  as  a  wave  off,  or  that  a 
fair  pass  reflects  20%  more  proficiency  than  a  bolter.  Rather,  categorical  analyses  were 
performed.  Two  categories  were  used:  grades  at  least  as  good  as  3.0  and  grades  at  least  as 
good  as  4.0.  The  former  can  be  viewed  as  distinguishing  between  satisfactory  and 
unsatisfactory  landings,  the  latter  as  distinguishing  excellent  landings. 

To  summarize  the  data,  86%  of  the  landings  were  at  least  satisfactory,  while  33% 
were  excellent  Total  pilot  experience  ranged  from  351  hours  to  4,501  hours,  with  an 
average  of  1,5 16  hours.  Flying  in  the  previous  month  ranged  up  to  46  hours,  but  averaged 
21  hours.  37%  of  the  landings  were  made  by  F-14s  and  63%  by  A-7s.  Night  landings 
accounted  for  26%  of  the  total. 

B .  ANALYSIS 

The  analysis  sought  to  determine  the  probability  that  a  landing  would  satisfy  a 
chosen  performance  criterion  (either  satisfactory  or  excellent)  as  a  function  of  the  pilot's 
flying-hour  history  and  other  factors.  Equations  of  the  following  form  were  developed: 

log(p(s)/[l  -  PC*)]}  =  a0  +  a1xHt  +  a2xH30  +  a3xN  +  a4xF  ,  (l)7 

where 

p(s)  =  the  probability  of  success,  either  a  landing  grade  of  at  least  3.0 
or  4.0 

Hc  =  career  flying  hours 
H30  =  flying  hours  in  the  previous  month 

N  =  a  dummy  variable  taking  the  value  1  for  a  night  flight  and  zero 
otherwise 


7  The  dependent  variable  in  this  equation  is  in  the  form  of  what  is  called  a  logit  transformation.  It  has  the 
characteristic  that  it  constrains  the  predicted  probability  of  success  to  be  between  zero  and  one.  It  leads 
to  the  probability  of  success  being  an  S-shaped  function  of  the  independent  variables.  The  coefficients 
in  Equation  (1)  can  be  estimated  using  maximum  likelihood  techniques. 
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F  =  a  dummy  variable  taking  the  value  1  for  an  F-14  flight  and  zero  for 
an  A-7  flight 

a0,  ar,  a2,  a3,  a4  =  coefficients  to  be  estimated. 

The  expectation  is  that  alt  which  reflects  the  long-term  impact  of  additional  flying 
on  performance,  and  which  reflects  the  short-term  impact,  are  positive.  We  expect  a3 
and  a4  to  be  negative,  since  it  is  harder  to  land  at  night  and  since  the  F-14  is  a  heavier, 
haraer-to-land  plane  than  the  A-7. 

C.  RESULTS 

Table  1  presents  the  results  of  estimating  the  coefficients  of  equation  (1).  Graphical 
representations  of  the  results  in  Table  1  are  presented  in  Figures  1  through  4.  Figures  1 
and  2  show  satisfactory  landings  and  Figures  3  and  4  show  excellent  landings.  For  each 
success  criterion,  one  graph  depicts  the  effect  of  changing  career  flying  hours  and  one 
depicts  the  effect  of  changing  flying  hours  in  the  last  month.  The  results  confirm  the 
hypotheses  about  the  links  between  flying  hours  and  aircrew  performance. 


Table  1.  Determinants  of  the  Probability  of  Meeting  Landing  Grade  Criteria 
for  A-7  and  F-14  Aircraft  —  Logit  Coefficients 


Satisfactory 

Excellent 

Landing 

Landing 

Cons'ant 

1.34 

-1.32 

(11.6)*"' 

(15.2)** 

Career  Flying  Hours 

0.00050 

.00024 

(9.06)** 

(8.51)** 

rru.;_^ 

±  IJ’Jilg  X  111 Ul  &  III  utc 

u.uo 

0.018 

Previous  Month 

(3.13)** 

(5.45)** 

Night  I^anding 

-0.619 

0.065 

(6.41)** 

(0.87) 

F-14  Flight 

-0.529 

-0.406 

(5.70)** 

(5.65)** 

Notes:  Numbers  in  parentheses  are  t-values.  Tout!  observations  were  4,351. 
**  Significant  at  the  .99  level. 


xy  Landing 


Figure  1.  Probability  of  Satisfactory  Landing  Versus  Career  Flying  Hours 


Flying  Hours  Previous  Month 

Note:  Career  flying  hours  at  mean. 

Figure  2.  Probability  of  Satisfactory  Landing  Versus  Flying  Hours  During 

Previous  Month 
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Probability  of  Excellent  Landing 


A-7  Day  Landings 
1 —  A-7  Night  Landings 
F-14  Day  Landings 
—  F-14  Night  Landings 


0  500  1  000  1  500  2000  2500  3000  3500  4000  4500  5000 

Career  Flight  Kioure 

Note:  Flying  hours  during  previous  month  at  mean. 

Figure  3.  Probability  of  Excellent  Landing  Versus  Career  Flying  Hours 


* —  A-7  Day  Landings 
A-7  Night  Landings 
I —  F-14  Day  Landings 
*—  F-14  Night  Landings 
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Flying  Hours  Previous  Month 
Note:  Carejr  flying  hours  at  mean. 

Figure  4.  Probability  of  Excellent  Landing  Versus  Flying  Hours 
During  Previous  Month 
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Both  recent  and  long-term  flying  experience  are  associated  with  better  performance. 
These  relationships  are  highly  statistically  significant  for  both  satisfactory  and  excellent 
landings.  Night  landings  were  less  likely  to  be  at  least  satisfactory.  However,  the  effect  of 
landing  at  night  was  not  significantly  different  from  zero  for  excellent  landings.8  F-14s 
received  lower  landing  grades  than  A-7s. 

In  quantitative  terms,  a  10%  decrease  in  the  number  of  flying  hours  would  have  the 
short-term  effect  of  increasing  the  number  of  unsatisfactory  landings  by  2.6%.  The 
number  of  landings  that  meet  the  excellence  criterion  would  decrease  by  2.5%.  If  the 
decrease  in  flying  hours  continued  indefinitely,  it  is  reasonable  to  assume  that  pilots  would 
have  10%  fewer  career  flying  hours.9  This  would  yield  a  further  increase  of  6.9%  in  the 
number  of  unsatisfactory  landings  and  a  further  decrease  of  2.4%  in  the  number  of 
excellent  landings.  Thus,  the  total  long-term  effect  of  a  10%  reduction  in  flying  hours  is 
estimated  to  be  roughly  10%  more  unsatisfactory  landings  and  5%  fewer  excellent 
landings. 

Most  of  the  total  effect  of  a  reduction  in  flying  hours  is  due  to  its  implications  for 
total  career  experience.  Performance  is  less  sensitive  to  short-term  changes  in  flying  hours 
than  to  long-term  changes.  This  is  important.  It  means  that  it  would  be  difficult  to  remedy 
problems  of  inadequate  performance  quickly. 

If  performance  is  allowed  to  degrade  because  of  inadequate  flying  hours  limiting 
overall  pilot  experience,  extremely  high  operating  tempos  would  be  needed  to  prepare  for 
an  emergency.  Such  operating  tempos  might  not  be  sustainable.  If  planes  are  flown  more 
intensively,  more  equipment  will  break  It  could  well  be  impossible  to  fix  the  failures  fast 
enough  to  keep  operating  tempos  at  the  desired  level.  Even  today,  aircraft  availability  is 
sometimes  a  limiting  factor  on  operating  tempo  in  the  Air  Force.  In  addition,  effective 
training  requires  use  of  bombing  ranges  and  air  combat  maneuvering  ranges  (and  other 
specialized  training  assets).  These  assets  are  heavily  used.  With  the  current  inventory  of 
ranges,  it  would  not  be  possible  to  meet  the  requirements  of  a  rapid  improvement  in 


8  One  experienced  earner  pilot  was  not  surprised  by  this  result  He  noted  that  there  were  fewer  distractions 
at  night.  He  acknowledged  that  it  is  more  difficult  to  land  satisfactorily  at  night,  but  felt  that 
satisfactory  landings  are  more  likely  to  be  excellent. 

9  In  fact  they  would  probably  have  even  fewer.  One  reason  pilots  stay  in  the  services  is  to  fly,  and  they 
feel  less  safe  flying  when  they  fly  less.  A  decrease  in  flying  hours  would  probably  reduce  retention  and 
thus  reduce  the  average  number  of  career  flying  hours  of  the  remaining  pilots. 
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training  readiness.  It  seems  that  we  could  save  lots  of  money  on  flying  hours  over  the  long 
term,  but  we  might  be  unable  to  prepare  adequately  for  combat.10 


10  Also,  the  cost  of  training  additional  pilots  to  make  up  for  higher  attrition,  or  the  cost  of  bonuses  to 
avoid  higher  attrition,  could  more  than  offset  the  savings  on  fuel. 
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HI.  ANALYSIS  OF  MARINE  CORPS  BOMBING 


A.  DATA 

Alone  among  the  services,  the  Marine  Corps  maintains  a  database  at  the 
headquarters  level  that  includes  noth  the  performance  information  and  the  information  on 
training  history  needed  to  examine  the  existence  of  statistical  relationships  between  flying 
hours  and  aircrew  performance.  This  information  is  part  of  the  Naval  Flight  Record 
Subsystem  (NAVFLIRS).  After  every  flight  pilots  fill  out  forms  describing  their  flight 
activity.  When  this  activity  includes  bombing  practice  on  instrumented  ranges,  the 
accuracy  with  which  bombs  were  dropped  is  included  in  the  report  NAVFLIRS  also 
includes  information  on  the  career  flying  history  of  pilots.11 

Data  were  obtained  on  23,000  flights  between  January  and  July  1987.  The  flights 
include  649  bombing  flights  for  which  information  on  bombing  accuracy  was  recorded. 
The  database  is  derived  from  bombing  runs  of  three  kinds  of  aircraft,  the  AV-8B,  F/A-18, 
and  F4-S.  Both  manual  bomb  deliveries  and  deliveries  in  which  automated  fire  control 
systems  were  employed  are  included,  and  every  bombing  flight  is  identified  as  either 
involving  manual  or  automated  bombing.  Rights  are  also  identified  according  to  where 
they  fit  into  the  training  syllabus  that  pilots  must  complete  to  remain  qualified.  It  was 
possible  to  differentiate  between  basic  syllabus  flights  and  advanced  syllabus  flights.  Table 
2  summarizes  the  Marine  Corps  data. 


The  range  of  career  experience  is  slightly  larger  than  it  was  in  the  carrier  landing 
analysis.  There  is  somewhat  less  recent  flying.  Almost  two-thirds  of  the  bombing 
deliveries  were  automated.  This  reflects  the  fact  that  the  vast  majority  of  AV-8B  and  F/A- 
18  deliveries  were  automated.  The  F-4S  does  not  have  an  automated  delivery  system. 


11  The  Navy  also  uses  NAVFLIRS,  but  when  practice  bombing  is  performed.  Navy  pilots  are  not  required 
to  report  bombing  accuracy  results  to  the  database.  Marine  pi'ots  are.  NAVFLIRS  has  only  been  in  use 
since  January  1987.  Before  that  time  a  similar  system  called  the  Flight  Readiness  Data  System 
(FREDS)  kept  track  of  flying-hour  histories,  but  not  of  bombing  accuracy. 
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Table  2.  Description  of  Data  Used 
in  Analysis  of  Marine  Corps  Bombing  Accuracy 


Variable 

Minimum 

Mean 

Maximum 

Miss  Distance  (feet) 

0 

83 

627 

Career  Flying  Hours 

226 

1,598 

5,875 

Flying  Hours  in  the 

Last  7  Days 

0 

4 

22 

Automated  Delivery8 

0 

0.64 

1 

In  Advanced  Part  of  Syllabus 

0 

0.24 

1 

AV-8B  Flight 

0 

0.53 

1 

F/A-l 8  Flight 

0 

0.17 

1 

F-4S  Flight 

0.30 

1 

8  The  last  five  variables  in  the  table  are  dichotomous.  They  either  take  the  value 
one,  showing  that  an  observation  has  the  indicated  property,  or  the  value  zero, 
showing  that  it  doesn't  The  mean  is  the  fraction  of  the  observations  that  have 
the  indicated  property. 

B.  ANALYSIS 

The  central  hypotheses  of  this  analysis  are  that  pilots  with  more  career  experience 
drop  their  bombs  more  accurately  and  that  greater  recent  flying  experience  is  associated 
with  more  accurate  bombing.12  Important  subsidiary  hypotheses  are  that  automated  bomb 
deliveries  are  more  accurate  than  manual  ones  and  that  experience — both  short-term  and 
long-term— plays  a  smaller  role  in  determining  the  accuracy  of  automated  deliveries  than  of 
manual  deliveries. 

Automated  delivery  systems  are  meant  to  increase  accuracy  and  to  make  bombing 
easier.  It  is  also  expected  that  bombing  accuracy  will  differ  according  to  the  type  of 
aircraft  The  newest  aircraft,  the  F/A-l  8,  is  expected  to  be  the  most  accurate  and  the  oldest, 


12  The  effect  of  recent  experience  on  performance  may  be  thought  of  as  working  through  the  training 
syllabus.  The  syllabus  is  put  together  so  that  skill  is  enhanced  as  a  pilot  progresses  through  it.  Thus, 
one  might  expect  greater  bombing  accuracy  on  advanced  syllabus  flights.  As  the  number  of  recent 
flying  hours  rises,  pilots  become  more  likely  to  be  in  the  advanced  part  of  die  syllabus.  Thus,  the 
flying-hour  program  can  be  expected  to  affect  the  degree  of  syllabus  completion,  which  in  turn  affects 
performance.  This  may  be  termed  the  recursive  form  of  the  short-term  experience  hypothesis.  An  • 

analysis  of  this  form  of  the  hypothesis  was  performed  and  is  referred  to  in  a  subsequent  footnote. 
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the  F-4S,  the  least  accurate.  These  hypotheses  led  to  the  formulation  of 
equation: 

M  =  bfl  +  bj  x  Hc  +  b2  x  H7  +  b3  x  A  +  b4  x  Hc  x  A  +  b5  x  H7  x  A  +  b6 
+  b7  x  F4  , 

where 

M  =  bombing  accuracy  as  measured  by  the  distance  by  which  the  bomb 
misses  its  target,  in  feet 

Hc  =  career  flying  hours 

H7  =  flying  hours  in  the  previous  seven  days15 

A  =  a  dummy  variable  taking  the  value  1  for  an  automated  delivery  and 
zero  for  a  manual  delivery 

AV8  -  a  dummy  variable  taking  the  value  1  for  an  AV-8B  flight  and  zero 
otherwise 

F4  =  a  dummy  variable  taking  the  value  1  for  an  F-4S  flight  and  zero 
otherwise 

b0,  bj,  b2,  b3,  b4,  b5,  b6,  b7  =  coefficients  to  be  estimated. 

The  coefficient  b7  measures  the  effect  of  additional  career  flying  hours  on  bombing 

performance,  and  is  expected  to  be  negative  —  better  performance  is  reflected  in  smaller 
miss  distances.  Similarly,  b2,  the  strength  of  the  short-term  influence  of  flying  hours  on 
performance,  should  be  negative.  We  also  expect  b3  to  be  negative,  picking  up  the  role  of 
automated  bombing  systems  in  improving  accuracy,  and  b6  should  be  positive  because  the 

AV-8B  is  expected  to  be  less  accurate  than  the  F/A-18  (the  only  model  of  plane  that  is  not 
explicitly  identified  by  a  variable  in  the  equation).  We  expect  b7to  be  positive,  because  the 

F-4S  is  expected  to  be  less  accurate  than  the  F/A-18. 

The  interpretation  of  b4  and  b5  is  somewhat  more  complicated.  These  coefficients 
measure  the  degree  to  which  flying  hours  improve  performance  differently  for  automated 
bombing  than  for  manual  bombing.  The  effect  of  additional  flying  is  hypothesized  to  be 
less  for  automated  bombing.  That  is  to  say  there  should  be  a  less  negative  impact  on  miss 


the  following 
x  AV8  (2) 


15  In  the  landing  grade  analysis  the  short-term  experience  variable  was  flying  hours  in  the  previous  month. 
The  seven-day  variable  was  far  more  successful  in  equation  (2)  than  the  previous  month  variable.  There 
is  no  intrinsic  reason  why  recently  honed  skills  ought  to  depreciate  at  the  same  rate  for  landing  and 
bombing.  It  is,  however,  interesting  that  the  bombing  analysis  was  far  more  sensitive  to  the  choice  of  a 
short-term  experience  variable  than  the  landing  grade  analysis  was. 
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distance.  In  this  case  b4  and  b5  will  be  positive.  The  effect  of  additional  career  flying 
hours  on  bombing  accuracy  for  automated  runs  is  measured  by  bj  +  b4.  The  effect  of 
additional  recent  flying  hours  on  bombing  accuracy  for  automated  runs  is  measured  by  b2  + 

b5- 

C.  RESULTS 

The  results  presented  in  Table  3  are  consistent  with  all  of  the  hypotheses  discussed 
above.14  They  are  presented  graphically  in  Figures  5  and  6,  Figure  5  depicts  the  effect  of 
changing  career  flying  hours,  and  Figure  6  depicts  the  effect  of  changing  flying  hours  in 
the  last  seven  days.  Each  graph  includes  five  curves,  representing  both  automated  and 
manual  bombing  on  the  three  kinds  of  aircraft  (the  F-4S  only  performs  manual  bombing). 
The  coefficients  of  both  the  long-run  and  short-run  flying  hour  variables  arc  statistically 
significant  in  the  expected  direction.  This  indicates  that,  for  manual  bombing,  additional 
flying  hours  improve  performance.  For  automated  deliveries  this  is  not  the  case.l5» 16 
When  the  coefficients  that  reflect  the  difference  in  the  impact  of  flying  hours  between 
manual  and  automated  flights  are  added  in,  additional  flying  hours  affect  bombing  accuracy 
very  little.  As  expected,  automated  deliveries  arc  more  accurate  than  manual  deliveries  and 
both  AV-8E  and  F-4S  deliveries  arc  iess  accurate  than  F/A-18  deliveries.17 


14  The  analysis  that  examined  the  recursive  form  of  the  short-term  experience  hypothesir  also  yielded 
positive  results.  Bombing  during  the  advanced  portion  of  the  syllabus  was  significantly  more  accurate 
and  pilots  who  had  flown  more  in  the  last  st  yen  days  were  significantly  more  likely  to  be  in  the 
advanced  portion  of  the  syllabus. 

15  This  does  not  mean  that  flying  hours  don’t  affect  pilot  proficiency  in  these  aircraft.  The  mission 
involves  more  than  delivering  bombs  accurately.  A  pilot  must  be  able  to  survive  ground-to-air  and  air- 
to-air  combat  to  be  able  to  deliver  his  bombs.  Performance  in  these  extremely  challenging  aspects  of 
the  mission  seems  likely  to  benefit  from  additional  flying  hours.  Reseaxh  into  the  relationship  between 
flying  hour  histories  and  performance  in  all  aspects  of  the  mission  should  be  putsued.  Air-to-air  combat 
is  treated  in  Section  IV. 

16  The  results  reported  here  are  quite  sensitive  to  the  specification  of  the  functional  form  hypothesized  in 
Equation  (2).  Alternative  functional  forms,  which  conform  slightly  better  to  the  data,  do  not  yield 
statistically  significant  results.  Alu  'nigh  we  emphasize  the  intuitively  appealing  results  in  Table  3,  we 
are  assembling  a  larger  database  to  va  idate  them. 

17  The  esdinated  equation  only  explains  about  one-fifth  of  the  variation  in  bombing  accuracy.  This  means 
that  the  equation  cannot  very  precisely  predict  where  an  individual  bomb  will  fall  based  on  the 
explanatory  variables  in  Equation  (2).  The  goal  of  this  paper  is  not,  however,  to  predict  the  location  of 
individual  bomb  deliveries.  Rather  it  is  to  estimate  the  effect  of  flying  hours  on  the  average  accuracy  of 
a  large  number  of  deliveries.  The  statistical  significance  of  the  coefficients  relating  to  the  impact  of 
flying  hours  indicates  that  it  is  adequate  to  this  task. 
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Table  3.  Determinants  of  Bombing  Accuracy  for  Marine  Corps  Aircraft 

(Miss  Distance  in  Feet) 


Independent  Variable 

Coefficient 

t-value 

Constant 

113.4 

10.10*** 

Career  Flying  Hours  (H£) 

-0.0094 

2.31‘* 

Flying  Hours  in  the 

Last  Seven  Days  (H?) 

-2.65 

2.07** 

Automated  Delivery  (A) 

-64.61 

5.62*** 

He  x  A 

0.0091 

1.80* 

H7  x  A 

3.13 

1.87* 

AV-8B  Flight 

20.96 

3.05*** 

F-4S  Right 

46.78 

4.57*** 

Note:  Adjusted  «  ,18. 

*  Significant  at  the  .90  level. 

**  Significant  at  the  .95  level. 
***  Significant  at  the  .99  level. 
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Flying  Hours  Prsvlous  Week 

Note:  Career  flying  hours  at  mean. 

Figure  6.  Bombing  Error  Versus  Flying  Hours  During  Previous  Week 

According  to  the  estimates  of  the  coefficients  of  Equation  (2),  if  flying  hours  were 
reduced  10%  for  a  short  period  of  time,  the  average  miss  distance  in  bombing  runs  would 
rise  by  about  1%  for  manual  bomb  deliveries.  If  the  reduction  continued  indefinitely,  an 
additional  increase  of  more  than  1%  would  be  incurred.  Once  again,  most  of  the  impact  of 
a  change  in  flying  hours  on  performance  appears  to  operate  through  its  effect  on  total  pilot 
experience.  A  pilot  with  3,000  career  hours  of  experience  on  average  can  be  expected  to 
place  bombs  24  feet  closer  to  the  target  than  a  pilot  with  500  hours. 

These  results  are  largely  consistent  with  uiose  obtained  in  a  1986  Air  Force  study 
[5].  That  study  examined  the  relationships  between  flying  experience  and  bombing 
accuracy  in  the  A- 10  and  F-16  aircraft.  It  found  both  long-term  and  short  term  experience 
effects  with  the  long-term  effects  more  pronounced  than  the  short-term  effects. 
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IV.  ANALYSIS  OF  AIR-TO-AIR  COMBAT  EXERCISES 


Approximately  every  eighteen  months,  each  Navy  F-14  squadron  in  the  Atlantic 
Fleet  participates  in  training  for  air-to-air  combat  at  Oceana  Naval  Air  Station.  The 
training — called  the  Fleet  Fighter  Air  Combat  Readiness  Program  (FFARP) — consists  of 
flying  a  series  of  exercise  missions  against  highly  trained  pilots  who  simulate  Soviet 
tactics.  Each  exercise  consists  of  a  control  phase  and  a  weapons  phase.  During  the  control 
phase  the  aircraft  crews  attempt  to  gain  and  maintain  radar  lock-on  and  position  themselves 
for  an  attack.  The  control  phase  ends  when  the  enemy  aircraft  is  sighted  and  visually 
identified.  The  weapons  phase  begins  when  a  weapon  is  fired  and  continues  until  all  Blue 
or  all  Red  aircraft  have  been  "killed."  Particularly  in  the  early  exercises.  Red  aircraft  may 
be  reactivated  in  order  to  give  the  Blue  crews  more  experience.  Reactivated  aircraft  return 
to  the  exercise  area  perimeter  before  rejoining  the  exercise. 
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(TACTS).  This  system  continuously  monitors  signals  from  all  the  aircraft  participating  in 
an  exercise.  It  tracks  their  positions,  observes  when  a  plane  "fires"  a  weapon,  and 
simulates  the  performance  of  the  weapon.  Every  firing  is  scored  by  the  system  according 
to  various  parameters,  such  as  speed,  attitude,  range,  acceleration,  and  altitude  of  the  firing 
and  target  aircraft 


The  F-14  is  a  two-seat  aircraft.  The  pilot  flies  the  plane  and  the  radar  intercept 
officer  (RIO)  is  resDonsible  for  keenine  track  of  and  tfllW.tinP  threatpnina  ynrrraft  Pith 
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crew  member  can  fire  missiles. 


Missions  are  flown  by  groups  of  two  or  more  "Blue"  F-14s  against  a  number  of 
aggressor  or  "Red"  aircraft.  Successive  exercises  become  more  complex  as  engagements 
progress  from  two  against  two  to  four  against  many  Red  aircraft.  Information  is  kept  on 
every  missile  shot  and  gun  firing.  In  addition  to  who  shot  and  whether  the  shot  hit  its 
target,  there  are  data  on  the  range  at  which  radar  contact  was  made  and  the  target  locked  on 
to.  The  ranges  at  which  the  target  was  observed,  visually  or  on  television,  and  identified 
are  recorded,  as  is  information  on  firing  range,  altitude,  elapsed  exercise  time  when  a 
weapon  is  fired,  and  the  missile  type  in  use  (for  missile  shots). 


A.  DATA 


Our  analysis  was  based  on  585  sorties  involving  1,820  missile  and  gun  shots.  Of 
these  1,820  attacks,  1,352  were  classified  as  valid  firings.  Each  valid  firing  is  an 
observation.  Attacks  by  aircraft  that  were  already  "dead"  or  against  "dead"  opponents  were 
eliminated  from  the  data  base.  A  total  of  967  valid  Blue  shots  were  recorded,  562  of  which 
were  successful.  Valid  Red  shots  numbered  384  with  131  successes. 

Average  pilot  experience  was  1,776  hours,  ranging  from  387  to  4,1 86.  RIOs  had 
less  experience,  averaging  1,332  hours  with  a  range  of  226  to  3,772.  Pilots'  flying  hours 
in  the  previous  month  were  between  11  and  38,  with  an  average  of  21.  RIOs  averaged  18 
hours  in  the  previous  montii,  with  a  range  of  4  to  38. 

The  definitions  of  variables  used  in  the  analysis  are: 

•  Performance  Variables 

-  Red  shoots  and  kills  (Yq) 

-  Red  shoots  and  misses  (Yj) 

-  Blue  shoots  and  misses  (Y2) 

Blue  shoots  and  kills  (Y3) 

•  Targeting  Effectiveness  Variables 

Lock  Range  Delta  (Rj):  The  difference  between  the  range  at  which  the 
crew  begins  the  exercise  and  the  range  at  which  radar  lock-on  is  made. 

-  Tally-Ho  Range  (R,):  Range  at  which  Red  aircraft  is  sighted,  either 
visually  or  on  TV. 

•  Crew  Experience  Variables 

-  Pilot's  total  career  flight  hours  (H^) 

-  Radar  Intercept  Officer's  career  flight  hours  (H^.) 

-  Pilot's  flight  hours  in  the  month  prior  to  the  beginning  of  the  FFARP 

<Hp30> 

-  Radar  Intercept  Officer's  flight  hours  in  the  month  prior  to  the  beginning 
of  the  FFARP  (H^ 

•  State  Variables 

-  Red  Odds  (Or):  The  ratio  of  Red  to  Blue  aircraft  when  the  shot  is  fired. 

—  Advanced  exercise  (E^J:  Equals  one  for  Competitive  exercise  or  more 
than  two  Blue  aircraft,  and  zero  otherwise. 
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-  Red  supersonic  (Sr):  Equals  one  if  opposing  Red  aircraft  is  supersonic, 
and  zero  otherwise.18 

Performance  variables  are  the  set  of  mutually  exclusive  and  exhaustive  outcomes  of  each 
firing.  We  hypothesize  that  performance  is  significantly  related  to  crew  experience. 
Targeting  effectiveness  variables  are  meant  to  capture  the  proficiency  of  the  crew  in  locating 
enemy  aircraft  and  in  positioning  their  own  aircraft  for  an  attack.  They  should  also  be 
affected  by  crew  experience.  State  variables  reflect  factors  over  which  Blue  has  little  or  no 
control  and  which  are  not  related  to  experience. 

B.  ANALYSIS 

Our  central  hypothesis  is  that  the  probability  of  a  given  outcome  depends  on  both 
the  recent  and  career  flight  experience  of  the  pilot  and  RIO,  and  on  the  exercise 
environment  The  exercise  environment  is  described  by  the  state  variables. 

Our  formulation  assumes  that  performance  is  determined  directly  by  targeting 
effectiveness.  Targeting  effectiveness — represented  by  Rj  and  Rt- — is,  in  turn,  influenced 
by  both  the  recent  and  career  experience  of  the  pilot  and  RIO.  Career  experience  of  the 
pilot — who  attempts  to  place  the  aircraft  in  the  most  advantageous  position  and  attitude  to 
assure  success — also  enters  directly  as  a  determinant  of  performance.  The  other  crew 
experience  variables  were  tried,  but  did  not  prove  important  in  this  role. 

The  model  is  defined  by  Equations  (3)  through  (5). 

ln  (Pi/p0)  =  aio  +  ajj  x  Hp,  +  ai2  x  Rj  +  aj3  x  R,  +  ^  x  Or  +  x  E^v  +  ai6  x  Sr ,  (3) 
i  =  1,  2,  3 

Ri  =  b0  +  bj  x  Hj*  +  b2  x  1^  + 1>3  x  Hp30  +  b4  x  Hr3a  (4) 

Rt  =  c0  +  c1xHpc  +  c2xHrc  +  c3xHp30  +  c4xHr30.  (5) 

where 

pj  =  the  probability  of  achieving  the  i*  outcome  (Yj) 
aik»  Cj  =  coefficients  to  be  estimated 

and  the  right-hand  variables  are  as  previously  defined. 

Equation  (3)  defines  the  multinomial  logit  (conditional  logit)  model,  which  is 
described  in  detail  by  Maddala  [6,  p.  41]  and  T.  Amemiya  [7,  p.  1516].  Maximum 

18  The  Red  supersonic  dummy  was  included  in  the  analysis  as  a  control.  The  coefficients  of  this  variable 
are  not  reported  because  of  the  potential  sensitivity  of  the  variable. 
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likelihood  estimates  of  Equation  (3)  were  made  us'ng  the  Newton  method  [8,  p.  150].  The 
coefficients  of  Equations  (4)  and  (5)  were  estimated  using  ordinary  least  squares.  The 
partial  derivatives  of  the  Pj,  i  =  0,  1,  2,  3  with  respect  to  the  right-hand  variables  in 
Equation  (3)  were  calculated  using  a  FORTRAN  program  written  by  Matthew  Goldberg  of 
IDA.  The  derivatives  of  the  pj  with  respect  to  the  individual  experience  variables  were  then 
computed  using  the  pi  partial  derivatives  from  Equation  (3)  and  the  estimated  coefficients  of 
Equations  (4)  and  (5). 

We  expect  the  coefficients  of  Equation  (4)  to  be  negative.  More  experienced  crews 
are  expected  to  lock  on  to  the  target  sooner.  Tally  -ho  range,  analyzed  in  Equation  (5),  is 
expected  to  depend  positively  on  all  of  the  experience  variables.  The  ability  to  spot  aerial 
targets  is  learned  behavior. 

Table  4  summarizes  our  hypotheses  concerning  the  partial  derivatives  of  the 
probabilities  of  a  Red  or  Blue  kill  with  respect  to  the  experience  and  state  variables.  We 
cannot  hypothesize  the  effect  on  Red  or  Blue  misses.  The  model  is  unordered  in  that  it  is 
not  clear  that  a  Red  miss  is  better  or  worse  than  a  Blue  miss.  One  might  speculate  that  it  is 
always  better  to  shoot  and  miss  than  to  be  shot  at  However,  a  missed  shot  may  give  away 

tVlP  clir\Af  ArV  tWCihAn  A**  oIIahi  tKa  rvnrKrvna«4-  ♦  < rw o  {>/v  i*nu*t  o  fmiAMiLI  a  ■•.AnaftAM 
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Table  4.  Hypothesized  Signs  ot  the  Partial  Derivatives  ot  the  Probability 
of  a  Red  or  Blue  Kill  with  Respect  to  the  Experience  and  State  Variables 


Independent  Variable _ 

Pilot  Career  Hours  (Hpc) 

RIO  Career  Hours  (!!„) 

Pilot  Hours  Previous  Month  (Hpjo) 
RJO  Hours  Previous  Month  (Hr30) 
Red  Odds  (Or) 

Advanced  Exercise  (E^v) 


_  Dependent  V  an  a  hie 

P  (Red  Kills  Blue)  P  (Blue  Kills  Red) 

+ 

-  + 

~  + 

-  + 

+  - 

+  - 


C.  RESULTS 

Table  5  shows  the  results,  in  partial  derivative  form,  of  estimating  the  coefficients 
of  Equation  (3).  The  model  was  estimated  with  each  observation  being  an  individual  shot. 
The  signs  of  all  of  the  partial  effects  arc  consistent  with  the  hypotheses  outlined  above. 
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The  targeting  effectiveness  variables  are  significantly  related  to  the  probability  of  a  Red  hit 
and  of  a  Blue  hit  and  both  ha\  the  expected  sign. 

Table  5.  Determinants  of  Performance  in  Air*to*Air  Combat 


Outcome 

Independent  Variable 

Red  Hits 
Blue 

Red  Miss 

Blue  Miss 

Blue  Hits 
Red 

Pilot  Career  Flight  Hours 

-2.56E  -  05 
(5.02)*** 

-7.72E  -  06 
(0.773) 

-2.93E  -  06 
(0.247) 

3.63E  -  05 
(2.87)*** 

Radar  Lock  Delta 

2.03E  -  03 
(2.94)*** 

4.47E-03 

(3.14)*** 

1.40E-03 

(0.806) 

-5.09E  -  03 
(2.70)*** 

Tally-ho  Range 

-4.15E  -03 
(2.80)*** 

-1.42E  -02 
(4.36)*** 

-2.15E  -04 
(.058) 

1.85E-02 

(4.61)*** 

Red  Odds 

1.74E  -  02 
(4.36)*** 

2.05E  -  02 
(2.10)** 

-1.37E  -  02 
(0.97) 

-2.42E  -  02 
(1.59) 

Advanced  Exercise 

4.64E  -  02 
(3.96)*** 

-6.91E-03 

(0.29) 

3.72E  -  02 
(1.27) 

-7.66E  -  02 
(2.45)** 

Number  of  Observations 

1,352 

Note:  Numbers  in  parentheses  are  t-values. 
**  Significant  at  the  .95  level. 

*•*  Significant  at  the  ,99  level. 


The  sigrs  of  the  state  variable  partial  effects  are  consistent  with  the  hypotheses 
stated  in  Table  4.  All  of  the  state  variable  coefficients  are  significant  at  the  .95  k  vel,  except 
for  the  effect  of  Red  odds  at  the  time  of  the  shot  on  the  probability  of  a  Blue  hit. 

Table  6  shows  the  partial  effects  of  crew  experience  on  the  targeting  effectiveness 
variables.  These  results  are  consistent  with  our  hypotheses  except  for  three  instances.  The 
long-term  experience  variables  are  not  significantly  related  to  the  distance  covered  between 
the  start  of  the  exercise  and  radar  lock-on.  We  cannot  explain  why  long-term  experience  is 
not  related  to  this  dependent  variable,  particularly  in  the  case  of  the  RIO.  The  fact  that  the 
RIO  short-term  experience  variable  is  not  significantly  related  to  tally-ho  range  is  more 
understandable.  The  pilot  has  a  greater  role  in  visually  sighting  the  target,  just  as  the  RIO 
has  primary  responsibility  for  the  radar  part  of  the  control  phase. 
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Table  6.  Determinants  of  Targeting  Effectiveness 


Independent  Variable 

Lock  Range 
Delta 

Tally-ho 

Range 

Constant 

2.74E  +  01 
(28.4)*** 

-1.26E  +  00 
(2.40)*** 

Pilot  Career  Flying  Hours 

5.57E  -04 
(6.34)*** 

RIO  Career  Flying  Hours 

9.56E-04 

(9.71)*** 

Pilot  Flight  Hours 

Previous  Month 

-9.9  IE  -02 
(2.82)*** 

1.58E-01 

(9.59)*** 

RIO  Flight  Hours 

Previous  Month 

-1.64E  -01 
(4.39)*** 

2.06E  -  02 
(1.15) 

Number  of  Observations 

1,352 

1,352 

R-Square 

0.02 

0.13 

Note:  Numbers  in  parentheses  are  t-values. 

***  Significant  at  the  .99  level  (two-tailed  test). 


The  determinants  of  performance  related  to  our  principal  hypothesis  are  shown  in 
Table  7.  All  of  the  signs  are  consistent  with  our  hypothesis,  and  the  effects  are  highly 
significant.  The  implications  of  these  results  can  be  more  clearly  seen  by  calculating  the  arc 
elasticities  of  performance  with  respect  to  each  of  the  experience  variables.  The  arc 
elasticities  were  calculated  to  show  the  expected  percentage  change  in  performance 
associated  with  a  10%  change,  from  the  mean,  of  experience. 


If  we  assume  a  10%  decrease  in  all  of  the  experience  variables  the  total  effect  on 
performance  would  be  a  4,8%  decrease  in  the  probability  of  a  Blue  kill  and  a  9.2%  increase 
in  the  probability  that  Red  kills  Blue.  Eighty-five  percent  of  the  expected  change  in  Red 
kills  is  attributable  to  the  effect  of  changes  in  pilot  flying  hours  (both  career  and  recent 
flying),  as  is  80%  of  the  expected  change  in  Blue  kills.  The  remaining  change  is  accounted 
for  by  decreases  in  RIO  career  and  short-term  flight  hours  in  approximately  equal  amounts. 
Pilot  career  flight  time  is  the  most  important  factor,  accounting  for  65%  of  the  increase  in 
Red  kills  and  42%  of  the  decrease  in  Blue  kills. 
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Table  7.  Full  Effects  of  Flying-Hour  Variables  on 
Performance  In  Air-to-AIr  Combat 


Outcome 

Independent  Variable 

Red  Hits 
Blue 

Red  Miss 

Blue  Miss 

Blue  Hits 
Red 

Pilot  Career  Flight  Hours 

-2.79E  -  05 
(5.56)*** 

-1.56E-05 

(1.57) 

-3.05E  -  06 
(0.257) 

4.66E  -  05 
(3.72)*** 

RIO  Career  Flight  Hours 

-3.97E  -  06 
(2.69)*** 

-1.35E  -  05 
(3.98)+** 

-2.06E  -  07 
(0.058) 

1.77E-05 

(4.17)*** 

Pilot  Flight  Hours 
Previous  Month 

-8.57E  -  04 
(3.36)*** 

-2.68E  -  03 

(4,48)*** 

1.05E-04 

(0.171) 

3.43E  -  03 
(4.57)*** 

RIO  Flight  Hours 
Previous  Month 

-4.18E-04 

(2.69)*** 

-1.02E  -  03 
(2.65)*** 

2.26E  -  04 
(0.753) 

1.22E  ”03 
(2.44)** 

Number  of  Observations 

1,352 

Note:  Numbers  in  parentheses  are  t- values. 

*  Significant  at  the  .90  level. 

**  Significant  at  the  .95  level. 

***  Significant  at  the  .99  level. 

The  effect  on  the  probability  of  Red  and  Blue  hits  of  each  of  the  experience 
variables  is  shown  graphically  in  Figures  7  and  8.  Each  experience  variable  is  allowed  to 
vary  over  its  observed  range  and  all  other  variables  are  held  constant  at  their  observed 
means.  These  graphs  illustrate  the  relatively  large  contribution  of  an  increase  in  pilot  flight 
hours,  especially  career  flight  hours,  to  a  decrease  in  Red  kills  and  an  increase  in  Blue  kills. 


23 


V.  SERVICE-SPONSORED  RESEARCH 
ON  FLYING  HOURS  AND  PERFORMANCE 


The  General  Accounting  Office  has  criticized  both  the  Air  Force  and  the  Navy  for 
failing  to  relate  flying  hours  to  measures  of  performance  [1],  [9].  Both  services  are 
currently  either  performing  or  sponsoring  research  into  the  performance  implications  of 
variations  in  flying-hour  programs.  This  chapter  provides  a  brief  overview  of  these 
efforts. 


A.  TACTICAL  AIR  COMMAND  (TAC) 

The  Tactical  Air  Command  has  recently  completed  a  quick  study  of  the  implications 
of  changes  in  the  flying-hour  program  for  aircrew  safety.19  The  analysis  examined  year- 
to-year  fluctuations  in  accident  rates  in  TAC  as  a  function  of  the  number  of  hours  flown 
during  a  period  from  the  eariy  70s  to  the  mid  80s.  A  strong  inverse  relationship  was  found 
between  the  two  variables.  Flying  hours  were  high  during  the  early  part  of  the  period,  then 
fell  in  the  late  70s  and  rose  in  the  80s.  Accident  rates  displayed  the  opposite  fluctuations, 
rising  in  the  late  70s  and  falling  in  the  80s.  A  quantitative  relationship  between  flying 
hours  and  accident  rates  was  derived. 


The  study  used  this  relationship  to  perform  a  cost-benefit  analysis.  The  savings 
that  accrued  by  reducing  flying  hours  were  compared  with  the  additional  costs  due  to  losing 


more  aircraft  and  pilots.  The  cost  of  pilots  reflected  the  high  cost  01  uicir  training,  i  ne 
study  found  the  cost  savings  from  flying  less  were  fully  offset  by  the  costs  of  additional 
accidents. 


TAC  is  conducting  a  much  more  detailed  study  of  flying  hours  and  performance. 
The  research  developed  task-oriented  performance  indicators  for  training  events  associated 
with  all  of  TAC's  tasks. 


Proficiency  was  related  to  flying-hour  histories  in  a  test  performed  in  two 
squadrons  in  1989.  Learning  curves  were  developed  for  both  experienced  and 


19  The  written  results  of  this  analysis  are  not  available  at  the  time  of  this  writing.  The  information  here  is 
based  on  discussions  with  TAC  personnel. 
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inexperienced  pilots  in  a  variety  of  tasks.  The  work  may  be  extended  to  additional  aircraft 
types. 

B .  STRATEGIC  AIR  COMMAND  (SAC) 

SAC  Headquarters  initiated  a  study  of  the  effect  of  variations  in  flying-hours  on 
performance  in  bombing  competitions  [10].  Several  measures  of  performance  were 
correlated  with  career  and  recent  flying  experience  for  aircrew  members.  The  results  of  this 
work  were  disappointing. 

No  consistent  relationship  between  performance  and  flying  hours  was  found. 
Many  factors  could  help  explain  this  result.  For  example,  the  crews  in  bombing 
competitions  are  selected  lfom  among  the  best  crews.  Flying  hour  histories  could  have 
little  relationship  to  proficiency  within  this  select  group,  but  still  be  important  for  the  wider 
population  of  aircrew  members.  Strategic  bombing  is  highly  automated.  As  the  Marine 
Corps  analysis  showed,  the  accuracy  of  automated  bombing  may  be  little  affected  by  flying 
hours.  Other,  more  tactically-oriented  parts  of  the  SAC  mission  may  be  more  affected. 
Also,  bombers  are  multi-person  aircraft  It  may  be  difficult  to  attribute  aspects  of 
performance  to  the  experience  of  individual  crew  members,  which  is  what  the  SAC  study 
tried  to  do.20 

SAC  is  continuing  work  in  this  area  that  draws  on  the  lessons  of  its  first  effort.  A 
structured  experimental  design  is  being  used.  Many  aspects  of  performance  arc  being 
scored  by  trained  observers.  Initial  results  are  encouraging.  In  addition,  a  research  project 
has  been  initiated  by  Systems  Research  and  Applications  Corporation  (SRA)  with  support 
from  the  Air  Force.  The  SRA  work  is  focusing  on  the  performance  of  tanker  crews. 

C.  MILITARY  AIRLIFT  COMMAND  (MAC) 

The  Air  Force  Human  Resources  Laboratory  (AFHRL)  is  sponsoring  a  program  of 
research  into  the  effectiveness  of  training  for  the  C-130.  A  principal  objective  is  to  evaluate 
a  new  program  of  contractor-managed  classroom  training,  but  the  AFHRL  personnel 
involved  are  also  supporting  efforts  to  investigate  the  importance  of  flying  hours  for 
performance  in  the  C-130. 


20  The  landing  grade  analysis  for  the  Navy  E-2C,  a  two-pilot  aircraft,  also  proved  unsuccessful.  There  may 
be  a  pattern  here. 
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D.  THE  NAVY 


The  Navy's  Aviation  Manpower  and  Training  Division  has  asked  the  Center  for 
Naval  Analyses  to  begin  a  study  of  flying  hours  and  performance.  A  long-term  effort  is 
proceeding.  The  plan  is  to  follow  several  squadrons  through  the  training  and  deployment 
cycle  to  develop  an  in-depth  understanding  of  what  performance  measures  should  be 
capturing  and  what  available  measures  actually  capture.  If  existing  data  are  felt  to  be 
inadequate,  recommendations  will  be  made  concerning  the  development  of  alternative  data. 
After  appropriate  measures  of  performance  have  been  decided  on  and  the  necessary  data 
gathered,  analysis  of  the  impact  of  flying  hours  on  performance  will  begin.  In  addition  to 
initiating  this  new  study  effort,  the  Navy  has  been  actively  helping  the  Institute  for  Defense 
Analyses  acquire  the  data  needed  to  perform  the  work  described  here. 


VI.  CONCLUSIONS 
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Existing  data  on  performance  can  be  used  to  develop  relationships  between 
aircrew  performance  and  both  long-term  and  short-term  experience  variables 
that  reflect  the  impact  of  variations  in  the  flying-hour  program. 

The  quality  of  Navy  carrier  landings,  the  accuracy  of  Marine  Corps  bombing, 
and  the  performance  of  Navy  aircrews  in  air  -to-air  combat  are  influenced  by 
the  number  of  hours  pilots  have  flown,  both  recently  and  over  the  course  of 
their  careers.21 


A  10%  reduction  in  flying  hours  is  estimated  to  increase  unsatisfactory 
landings  by  10%  and  decrease  excellent  landings  by  5%  over  the  long  term. 
Such  a  reduction  is  also  estimated  to  decrease  bombing  accuracy  by  about  2% 
for  manual  bomb  deliveries.  Additional  flying  hours  have  not  been  found  to 
affect  the  accuracy  of  automated  bomb  deliveries.  A  10%  decrease  in  flying 
hours  for  fighter  aircrews  is  estimated  to  increase  the  probability  that  red  kills 
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Research  to  date  has  focused  on  only  a  few  aspects  of  the  jobs  assigned  to 
aircrew  members — bombing  accuracy,  landing  proficiency,  and  flying  safety. 
Our  analysis  of  air-to-air  combat  has  expanded  both  the  range  of  tasks  and  the 
number  of  crew  members  entering  the  analysis. 


The  long-run  effects  of  flying  hours  on  performance  appear  to  be  more 
quantitatively  important  than  the  short  run  effects.  This  is  illustrated  in  Table 
8.  It  means  that  in  an  emergency  it  would  be  difficult  to  remedy  an  inadequate 
level  of  training  The  availability  of  aircraft  and  of  training  ranacs  would 
further  constrain  the  ability  to  improve  training  readiness  quickly. 


There  was  some  concern  that  the  apparent  importance  of  career  experience  might,  to  some  extent,  be  a 
statistical  artifact.  This  could  be  the  case  if  good  pilots  are  more  likely  to  stay  in  the  service  and 
accumulate  a  large  number  of  flying  hours.  To  examine  this  hypothesis  the  analysis  of  air-to-air  combat 
was  redone  with  only  pilots  who  were  believed  to  be  in  the  first  term  of  service.  The  results  of  this 
reexamination  did  not  differ  much  from  the  results  reported  in  Section  IV.  The  interpretation  that  more 
(lying  causes  better  performance  (rather  than  the  reverse)  seems  to  be  correct. 


Table  8.  Impact  of  10-Percent  Cuts  In  Flying-Hour 
Variables  on  Performance 


Performance  Measure 

Career  Flying 

Recent  Flying 

Total 

Unsatisfactory  Landings 

6.9% 

2.6% 

9.5% 

Bombing  Miss  Distance 
Air-to-Air  Combat 

1.5% 

1.0% 

2.5% 

Probability  Red  Kills  Blue 

6.3% 

2.9% 

9.2% 

Probability  Blue  Kills  Red 

-2.6% 

-2.2% 

-4.8% 

VII.  FUTURE  RESEARCH 


A  start  has  been  made  in  linking  flying  hours  to  indicators  of  aircrew  proficiency. 
Analysis  of  objective  data  has  shown  that,  in  selected  cases,  reductions  in  flying  hours  will 
lead  to  measurable  degradations  in  mission  performance.  Still,  there  is  much  work  left  to 
be  done. 


Research  to  date  has  focused  on  just  a  few  aspects  of  the  jobs  assigned  to  aircrew 
members — bombing  accuracy,  landing  proficiency,  flying  safety,  and  air-to-air  combat.21 
Many  critical  parts  of  the  missions  entrusted  to  military  aircrews  have  not  been  analyzed. 
These  include  in-flight  refueling,  helicopter  operations,  and  low-level  cargo  drops. 
Existing  evidence  can  be  used  to  help  justify  flying-hour  programs,  since  it  shows  that 
flying  hours  make  an  objectively  measurable  difference,  but  it  cannot  show  that  they  make 
a  difference  everywhere. 

in  addition  to  the  omission  of  some  missions  from  the  body  of  successful  flying- 
hour  research,  the  role  of  flying  hours  in  enhancing  the  level  of  performance  in  large 
aircraft  (e.g.,  tankers,  bombers,  and  cargo  aircraft)  has  not  been  demonstrated 
quantitatively.  The  task  of  attributing  aspects  of  performance  to  the  experience  of 
individual  aircrew  members  is  obviously  harder  in  a  crew  made  up  of  many  people. 


IDA  plans  to  focus  its  continuing  research  program  on  the  gaps  in  the  flying  hour 
literature.  The  training  research  being  done  by  MAC  is  expected  to  result  in  the 
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IDA  IS  maintaining 


contact  with  the  MAC  researchers  and  has  assembled  a  performance  and  flight  hour  data 
base  for  an  analysis  of  C-130  drop  accuracy.  An  effort  will  be  made  to  gather  data  for  an 
analysis  of  helicopter  performance  for  at  least  one  of  the  services. 


Over  the  next  few  years  it  is  realistic  to  hope  that  illustrative  relationships  between 
flying  hours  and  performance  will  be  developed  that  cover  most  important  aircrew  missions 
and  most  types  of  aircraft.  These  relationships  should  provide  a  broad-based  justification 
of  the  importance  of  flying  hours  in  producing  and  maintaining  aircrew  readiness.  One 


21  In  addition  to  the  recent  work  at  TAC,  the  Naval  Safety  Center  has  performed  research  that  showed  a  link 
between  career  flying  hours  and  accident  rates  [1  lj. 
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must  be  careful,  however,  not  to  expect  too  much  from  these  relationships.  There  are  two 
valuable  products  they  are  not  likely  to  provide. 


The  kinds  of  relationships  developed  in  this  paper  are  not,  by  themselves,  adequate 
for  the  task  of  determining  how  many  flying  hours  are  enough.  There  are  two  reasons  for 
this.  First,  the  estimates  of  the  quantitative  relationships  between  flying  hours  and 
performance  that  they  provide  are  not  veiy  precise.  They  are  adequate  for  giving  us 
confidence  that  a  positive  relationship  exists  between  flying  hours  and  performance,  and 
they  provide  a  plausible  estimate  of  the  magnitude  of  the  relationship,  but  the  actual 
magnitude  of  the  relationship  is  still  quite  uncertain.  Second,  there  is  usually  no  indication 
of  what  level  of  performance  is  necessaiy.  Is  it  adequate  for  87%  of  carrier  landings  to 
receive  grades  of  3.0  or  above,  or  is  90%  an  appropriate  goal?  What  difference  does  it 
make  for  the  operation  of  a  carrier  battle  group  in  either  peacetime  or  war?  Without 
answers  to  questions  like  these,  even  precise  relationships  between  flying  hours  and 
performance  cannot  tell  us  how  many  flying  hours  should  be  budgeted  for. 


In  addition,  information  on  relationships  between  aggregate  flying  hours  (either 
long-term  or  short-term)  and  performance  provide  little  guidance  about  the  appropriate 
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which  particular  tasks  are  practiced  in  order  to  minimize  the  number  of  flying  hours  needed 
to  develop  overall  mission  proficiency.  This  kind  of  insight  depends  on  doing  analysis  at 
the  task  level.  The  TAC,  MAC,  and  CNA  research  efforts  all  have  the  development  of 
such  insight  as  one  of  their  goals.  Successfully  performing  research  of  this  type  for  even 
one  kind  of  aircraft  requires  much  more  attention  to  data  development  than  does  research 
oriented  to  demonstrating  the  overall  value  of  additional  flying  hours.  It  very  much 
deserves  to  be  pursued,  but  quick,  wide-ranging  payoffs  should  not  be  expected. 


Research  on  flying  hours  and  performance  should  follow  a  two-pronged  path. 
Work  to  show,  in  rough  terms,  how  much  additional  flying  hours  enhance  performance  for 
a  wide  range  of  aircraft  and  missions  should  yield  useful  results  relatively  quickly — if 
adequate  resources  are  applied  to  the  area.  At  the  same  time,  analysis  of  task-oriented 
performance  data  could  yield  new  insights  into  the  proper  design  of  aircrew  training 
programs  over  a  longer  period. 
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